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SUBJECT: CAPTION SURFER PATENT PROPOSAL- 



AMENDMENT 



Joe, 



Following your receipt of the original 
we discussed it with HAL-Lega]» Davfi 
attached is an amendment to that *~ 
details regarding the features and 




^ atent proposal for this concept, 
ningham. At your and his request 
roposal with further, deeper 
of Caption Surfer. I did not 



specifically copy David on this cover memo as 1 thought you would want to 
review first and then forward via your office to David's attention. Induded as 
a separate pdf file is a copy of thelmProposal with two witnessed and 
understood signatures. 



In addition to the deeper detail discussion, I have also amended some of the itemized 
elements on our intexnal patent application form. Those itemized ameaidments 
immediately follow tins cover page as page 2-7 + slides 8-22 in JBle CS_EXAM.ppt. 




Microsoft's TV PAK web access is a member onfy site requiring a lbgin ID and 
password. When I emaHed to Microsoft and asked how to become a member 
they responded with an email advising that since I'm with Hitachi ( based on my 
email address and information I supplied to them (work address, phone, ^a) I 
would be allowed access to their Microsoft's email included the ID arxl 
password so I was able to obtain the attached exhibit ( If s why I never asked you 
to approve an NDA with Microsoft for ttiis subject! ). I have no idea how to 
handle such confidential documents in a patent disclosure application so I leave 
that to your and David's expertise. 

Rnal point is a group of color Powerpoint slide printouts attached shovwng one 
example of how thfe concept would wori^ from the customer's perspective. I also 
have the electronic Powerpoint presentation (including animations) and have 
supplied to you via email. 
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4. Please explain background drcmnstances. 




5- Please explain the objects^ features-^..,'.., 

Qrigiiial proposal did not detail sufSciently the "objecf * ( puxpose and 
operation ) of tbe invention. Further details are included as noted below. 

OBJECT; please see page 4 

TCCEINICAL DESCRIPTION: please see p^es 5-7 

6. DESCRBPHON of "BEST'* moi>e 

■PLEASE SEE 8 - 22 and ELECTRONIC EXAMPLE 

CS_EXAM,pptf 
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OBJECT of INVENTION: 
Purpose of invention is to allow TV viewer to locate information using keywords selected 
from U.S. closed caption data services commonly in iise "by today's television, movie 
studio ( VCR & DVD rentals ) or virtually any other data source supplying EIA-608 
compliant VBI data streams to a video display equipped with a closed caption decoder 
and Hois invention. 

The viewer can pick a word ( or words ) fix)m the TV*s on-screen closed caption display 
accompanying many television, VCR, DVD, satellite, cable or other program sources. 
The selection of the woxd(s) on the TV screen is accomplished using a "cursor^ style or 
tbumhstick or joystick type remote control- 
Once the woid(s) has/have been selected, the invention then "finds** those words in the 
TVs closed caption display memory; converts them to a format suitable for input to any 
one of many Internet browsers ( or other data locating means ), The browser ( or other 
data locating means ) letoms with data ( from the bttemet or other data source ) that most 
closely match the keyword search temtis "selected" by the customer from the closed 
caption on-scrccn display. 

The ^information'' ( as an easy to understand example ) conld be from fte Internet bat 
this invention does not restrict itself to just the Internet The information source being 
accessed by this invention could also be local ( built-in to the TV ) or a corporate, or 
library-based source of information. 

The attached Powerpoint slides depict one example of this invention's purpose to locate 
information relevant to the video program being watched. These slides represent only 
one example and do not mean to limit the scope of the intended invention. 
Accompanying this p^er desciiption is a separate electronic jSIe CS JEXAM.ppt of the 
Powerpoint slide presentation including animations showing this ^^typical" operation. 

Some other applications would include searching a 'Svarehouse" of video or other 
material with closed cEg>tion data in order to find warehouse material matcfaii:^ the 
keyword or ^'search tcrmCs)" suhmrtted by the user. In this materially dxi^xent 
application of tiie invention, the user would supply tiie search term or ke3rword(s) by 
keyboard inpnt and then the invention would search each program's closed c^ytion data 
for word(s) matching the user supplied keyword(s). 

The Powerpoint example shov/s an Intemet browser ( Microsoft's IE 5 ) used as the 
"search engiD^" but that is only one of many ways to implement a seardh function of 
remote or local data sonrces or ^'warehouses*'. The selected wo(rd(s) from the closed 
caption data stream could be converted to support many different types of '^search 
means" with ^3propriate interface. A further example ofa DVD-ROM (or RAM) 
"jukebox" conaining of several ( or a hundred ) DVD-ROM discs able to be brou^ "on- 
line" by medianical or electronic means. If these DVDs contain closed caption data 
encoded in compliance with EIA-608, this invention can seardi and catalog them. 
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TECHNICAL DESCRIPTION OF INVENTION: 

This invention is biuit lipon existrag ijaidware and software caminon in tpday's televisicMi 
receiver called CLOSED CAPTION DECODING, There is a standard developed by the 
EIA and authorized by the FCC for this data service using line 21 of a television signal*s 
vertical blanking interval (VBT). The £?>plicable standard is HLA-608A ( cunenfly being 
revised to B ). The applicable FCC authorization is provided in the Code of Federal 
Regulations (CFR 47,119) governing unplementation of EIA-608 in U.S. TV receivers 
and U.S. program caption encoding by content providers. 

Anyone can tunc to most any TV program today and encounter closed ce^ons by 
enabling that feature on the TV ( the feature is mandated by the CFRs in U.S. TVs of IS'* 
and larger since 1 993 ), It can be observed by watching such a TV and program, that . 
there already exists a means for encoding program captions. The standards ( EIA and . 
CFRs ) noted above also provide for a minimum level of stq)port with respect to character 
sets, colors and control codes to effect the on-screen location and s^pearance of the 
closed coitions. The standards also stipulate the method of encoding these characters and 
control codes in a manner consistent ftom channel to channel ( among TV stations ) and 
among content providers. This consist^cy guaarantees Ihe imiform transrnission and 
reception/decoding of captioned programs throughout the U.S. 

To siqpport the standard, many seimconductor manufacturers have / are producing 
microcontroUers ( micons ) with the hardware and software to implement the decoding 
and di^Iay of closed captions in a TV receiver. These same micons also typically handle 
the infia-red(IR)conmiands issued by a vievi^'s hand held remote controL Idmy 
remote controls today use a "curscM*' or joystick style of moving about on the TV's screen 
in order to allow the user (viewer) to turn on or off or otherwise adjust features built-in to 
the TV ( such as opening an on-screen menu to enable closed caption display ). 

This invention adds additional software to the TV's micon allowing it to provide the 
essential functions of the invention, namely: 

• enable the display of closed captions when instructed by the user 

• '^eze" the scrolling of on-screen captions when instructed by th^ 

• present on-screen to the user a ''cursor^ or other means allowing the wcr to 
*^oint" to a word or words within the frozen closed caption display area on- 
screen 

« relate the on-screen selected word(3) to closed cs^tion data stored in the mioon's 

closed cation buffer memoxy 
9 identify from that relation the specific word(s) in ti^ closed caption buffer 

• copy the selected bufier memory locations to anotiier new buffer 

• strip away the control codes before/after the new bujBer's contents 

• ourtput the new buffer's processed data to an gqapropriate irrterfece 

• manipulate the inter&ce*s coixtrols means to cause a search of that intec&tce's 
connected data sources for content containing/matching tiie new buffer's content 

« display the result of the interface's search results in list or table form 
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Page 6 



In block diagram fann below is a ''typical" implementatioii today of the U,S. standard 
for closed caption decoding with a brief description next to each block. 

video signal on-screen dtsptey 

contalr^ng clos^-»= i_r I— 4 ceo buffer )— 4 osDOinPUT U-».<OSD) grapWcs 
option data appll&d i_ ] ^ I 1 I i H 1 output to videp 



to 1V3 on-boaid dtsptay 
microcorrtmllar 

VIDEO SIGNAL contains captions encoded per EIA-608 

"SLICER" is naicon hardware and software that locates line 21 in the video signal 
and removes or ^slices" &e data &om that line. Sliced' data is put m micon's RAM 
"PARSER" examines title data sliced from line 21 to deteimine if it is valid caption 
data ( transmitted on line 21 in packets called CCl or CC2 or TI or T2 ) 
IF sliced data is valid caption or text data it is placed into micon RAM called tiie 
"CCD BUFFER" 

( there are OTHER types of data provided via line 21 such as "VCHIP" data v*rich 
is NOT displayed on-screen to the viewer and NOT conadered in this invention ) 

"OSD OUTPUT* is hardware and software rathe micon fliat is pgreprogrammed to 
understand the control codes embedded in the caption data stream and position 
the c^ytion data accordingly on the TV's screen ( in a manner similar to the 
on-screen TV menu system ) 



• EIA'-608 contains many control codes providing a rich set of options for 
presenting caption data on-screen such as: 

o Positioning tiie black background caption 'Svindow^ on-screen 
o POP ON or PAINT ON "window" styles 

Q Number of rows to present and scroU attributes of the caption window 
o Special characters to denote that music is being played as well as other 
special characters 

To implement this invention's essential function it needs to control the hardware and 

software mentioned above to: 

« cause the caption feature to be enabled ( if rt is not already enabled ) 
0 ''freeze** the contents of an on-scxeen c^gition window when the viewer sees a 
word tihey would like to select 

( this prevents subsequent caption data and control codes fiom overwriting or 
erasing the on-screen display as well as the contents of the CCD BUFFER ) 

• activate the TV's on-screen cursor "pointer*' ( or generate one if not provided ) 
» move the on-screen ^pointer* in response to viewer's commands as indicated by 

keystrokes on the TV' s remote ccmtrol 

• provide ameans ( a specific remote control key stroke ) to signal the invration 
that the vierwer has positioned the pointer over a word of interest 

« review the CCD BUFFER'S contents and exanaine control codes and character 
data to detemnne based on screen pointer location which word in the buffer 
corresponds to the word "pointed to" by viewer 
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copy tbat word to a new 'Svorking'* buffer ( RAM ) in order to strip away any 
remaining control codes ( and in tbe event the viewer points to a special character 
such as the clef symbol for music ) advise the viewer accordingly via the TV's 
on-screen menu system that a search for xxxxxxx ( the selected word ) will be 
made ( or that a selection error has been made ) 

output the working bxifFer*s processed data to one of many "interfaces" to be 
coupled to an appropriate means for searching the connected database or 
'Varehouse" ( Internet search or other source material )- 

( in this invention "interface" could mean to provide die selected word as data to a 
PC OT oflier device via USB, IEEE1394, RS-232, 12C, wireless, cellular, EtiberNet, 
HomePNA, etc^ etc.) 

display the interfece's returning result as a list or table for the user's review and 
passible further action 

( in the included Powerpoint example slides the *'interface" is an Internet capable 
browser built-in to the TV but tihds is not the only 'InteTface" possible in order to 
^tisiy the intent of this invention; it is one of the simplest to imderstand. It is 
passible via additional software and or hardware to constuct a means of interface, 
result display and further action using the TV's own built-in menu system or 
another device^s inter&ce with appropriate application software and/or haidware 
to provide the result and further action functions ). 



From the above description it is obvious that this invention builds rq;>Qn existing 
haidware and software whhin current television receivers today. By adding only 
software most TVs today would be able to provide this inventions fimctions accept 
for the search interface and resuh interface fhncticHis ( since most TVs today do NOT 
have Internet browsers buih-in ). 

While TV hardware and software is very arcane and specific to eadi TV 
manufacturer's design group^ a fuUy fimcdonal implementation of fids invention 
could be realized quickly on a PC-based platform by adding to ANY standard PC the 
fbllowii^: 

« a TV toner card which, supports closed caption decoding 

« application software which performs the fimctions of tins invention by lea^ng the 
TV tuner card's CCD BUFFER; allowing the customer to move the PC's mouse 
to select; performing the same processing of the CCD BUFFER'S data versus PC 
display mouse cursor location on-scxeen to identify the selected word; feedisg 
that word ( aftw strij^ing away control codes, eta ) to the PC's Internet browser. 

An additional feature of this invention is it's ability to "look" for URLs 
( http'yAvww>xxx.yyy or www,xxx.yyy ) sent via flie ceytioning service of line 21 . When 
found in the CCD bufier, these could be AUTOMATICALLY copied and stcared as a list 
for the customer to review and use as search terms to his/her lHx>w5er at a later time* 
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